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Medical examination method, system and computer program for stereoscopic viewing of 3d 
medical images 



29 - °s. m 



The invention refers to a medical examination method of visualizing an ^ 
internal hollow organ of a subject based on a volumetric scan thereof, said method 
comprising the step of: 

a) Reconstructing a three-dimensional image of the inside of the hollow organ. 



Such a method is known in the art and forms the basis for a number of 
computer programs designed by different experts in the field providing a virtual medical 
examination technique called "virtual endoscopy". Based on a volumetric scan of a patient, 

10 for instance generated by means of Computed Tomography, a data model is created from 
which three-dimensional endoscopic images are reconstructed by means of known three- 
dimensional reconstruction techniques. A 3D path is defined through a tubular structure or 
hollow organ of interest after which the inside thereof is visualized by moving a virtual 
camera along this path and calculating 3D endoscopic images as seen from view points lying 

15 on the path. Such computer programs offer a medically skilled person an opportunity to 

examine the internal organs of the patient without the need for invasive examination like true 
endoscopy. The thus reconstructed 3D endoscopic images can for instance be evaluated on a 
computer by a medically skilled person for diagnosis. 

The known method has the disadvantage that although the resulting 3D images 

20 are a true representation of the inner structure of the hollow organ the interpretation thereof 
remains difficult. In practice it appears for instance to be difficult for a user to keep track of 
the orientation of the virtual camera during interactive movement thereof. Furthermore the 
wide-angle view of the lens of the virtual camera appears unnatural to a human observer and 
thus hampers the interpretation. 

25 

It is an object of the method according to the invention to provide a method of 
the type as described above that provides a more natural view. 
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point that differs from the first view point; 

d) Combining the left eye image and the right eye image into a pair to form a 
stereoscopic image; and 

e) Showing the stereoscopic image using stereoscopic imager means. 



images the user sees the images in a more natural way. Stereoscopic images provide 
additional information, such as depth information and height information of the images. 
Using this information the user is for instance able to better judge relations between objects 
and/or to clearly see curvatures and/or height differences allowing him or her to distinguish 
1 5 between flat or inclining areas. As a result thereof the observer will be able to interpret the 
patient data more accurately and has a greater chance of arriving at the correct diagnosis. 



images, which is known per se in the field of computer graphics. A method of this type is for 
instance described in US patent 6,01 1 ,581. Contrary to the method according to the invention 
20 the known method is however not part of a method for performing virtual endoscopy. 



wherein step a) further comprises the step of: Defining a view path through the hollow organ, 
the method is characterized in that, for each image the first view point lies on a first line and 
the second view point lies on a second line, which first and second lines extend essentially 

25 parallel to the view path, each on one side thereof The user now follows the same path 
trough the hollow organ as during performance of virtual endoscopy in the known 
"monoscopic mode", but sees the 3D images in a more natural stereoscopic view. In support 
of this embodiment the image for each viewpoint needs to be calculated separately. 

In a second preferred embodiment of the method according to the invention, 

30 wherein step a) further comprises the steps of: 

- Defining a view path through the hollow organ; and 

- Reconstructing the three-dimensional images as seen from view points lying 
on the view path, the method is characterized in that, at least the first or the second view 
point lies on the view path. 



10 



By providing stereoscopic images of the hollow organ instead of monoscopic 



It is noted that steps b) through d) refer to the generation of stereoscopic 



In a first preferred embodiment of the method according to the invention, 
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The amount of images to be calculated is equal to the amount of images 
necessary for the first embodiment However, when the stereoscopic viewing technique is 
used in addition to the known monoscopic viewing technique, advantageously, for each 
stereoscopic pair only one additional image needs to be calculated. The monoscopic images 
5 for view points on the view path that are already calculated for the known monoscopic 

viewing technique can now be used as either the left or the right eye image of a stereoscopic 
pair. Hereby the overhead computations necessary to implement the method according to the 
invention are thus effectively minimized. 

In a third preferred embodiment of the method according to the invention, 
10 wherein step a) further comprises the steps of: 

- Defining a view path through the hollow organ; and 

- Reconstructing the three-dimensional images as seen from view points lying 
on the view path, the method is characterized in that, both the first and second view point lie 
on the view path. 

15 By choosing the first and second viewpoint cleverly now effectively in this 

third preferred embodiment the number of images to be calculated is equal to the number of 
images to be calculated for the known monoscopic method. The method according to the 
invention may also be implemented in addition to the known monoscopic method without 
any necessary overhead computations. Use can then be made of images that are already 

20 calculated for viewpoints on the view path for the known monoscopic method. The already 
calculated images for viewpoints on the path are most efficiently alternately used as first or 
second viewpoint. 

In a further preferred embodiment of the method according to the invention 
the distance between the first and the second viewpoint is essentially one or more 
25 millimeters. The resulting view appears to be most natural for a human observer. 

Preferably the view direction in the first and the second viewpoint is 
essentially parallel. Again the objective thereof is to simulate the normal view of a human . 
observer, thus facilitating a correct interpretation of the imaged data, which in turn will 
greatly enhance the chance that the correct diagnosis is made. 
30 According to another preferred embodiment of the method according to the 

invention step e) further comprises the steps of: 

- Showing the left and right eye image forming a stereoscopic image with 
different modification; and 



i 
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- Arranging the stereoscopic imager means such that the left eye image is 
passed to the left eye and the right eye image is passed to the right eye. 

Different modification of the left and right eye images in combination with 



suitable stereoscopic imager means is a practical way to accomplish stereoscopic vision. 
5 Various techniques hereof are known per se in the relevant art. In a first detailed embodiment 
use is made of so-called "passive" stereoscopic imager means. The left and right eye image 
of a stereoscopic image is shown with different polarization; and the stereoscopic imager 
means are provided with correspondingly differently polarized viewing means for 
respectively the left and right eye. In a second detailed embodiment the left and right eye 

1 0 image of a stereoscopic image are shown with different time-multiplexation. So-called 

"active" stereoscopic imager means are used with different viewing means for the left and 
right eye that are to be activated separately by a control unit based on corresponding time- 
multiplexation signals. In order to provide a geometrically correct image the viewing means 
are preferably incorporated in a head-mountable display. 

1 5 According to another preferred embodiment of the method according to the 

invention the stereoscopic imager means comprise a lenticular screen. Lenticular screens are 
known per se in the relevant art. Advantageously these screens are able to show a 
stereoscopic image as such without the need for an observer to use or wear any assisting 
means. 

20 The invention also refers to a system for visualizing an internal hollow organ 

of a subject based on a volumetric scan thereof, which system comprises means for carrying 
out the steps of the method according to the invention. 

The invention further refers to a computer program to carry out the method 
according to the invention. 



25 
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Fig. 4 schematically illustrates a second preferred embodiment of the method 

according to the invention; and 

Fig. 5 schematically illustrates a third preferred embodiment of the method 

according to the invention. 



In all figures equal objects are denoted with equal reference numerals. 

In general the method according to the invention refers to virtual medical 
examination techniques for examination of a subject, which is usually a human patient, but 
10 can also for instance be an animal. Said techniques allow an inner view of hollow structures 
of the subject, e.g. organs, blood vessels, etc., by means of computer graphics. A virtual 
camera is placed in a three-dimensional data volume representing (part of) the subject The 
method according to the invention will now be described according to a preferred 
embodiment, which relates to virtual endoscopy performed on the colon of a human patient 
15 In order to require the 3D patient data several known medical examination 

techniques can be used to acquire a volumetric scan, such as Computed Tomography (CT) or 
Magnetic Resonance Tomography (MR). The 3D data are visualized by means of known 
three-dimensional reconstruction techniques. For this purpose different suitable volume 
rendering techniques are known in the field of computer graphics. Preferably use is made of 
20 iso-surface volume rendering techniques, which are for instance described in the article "Iso- 
surface volume rendering 11 , by M.K. Bosma et aL, Proc. of SPBB Medical Imaging f 98, vol. 
3335, pp 10-19. Thus a virtual environment is created that simulates endoscopy. 

Fig. 1 shows a flow diagram presenting an overview of the steps of the method 
according to the invention. All of the steps will be separately discussed below. 
25 Method steps: 

Step 10: Reconstructing a number of three-dimensional images of the internal 
surface of the hollow organ; 

A variety of visualization techniques are available to the person skilled in the 
art to simulate a three-dimensional view of the colon through which a user may navigate. 
30 Several examples include: 

- the "unfolded cube" technique, wherein from each view point six views of 
the colon wall are calculated in mutually perpendicular directions, which views are projected 
onto the walls of a cube, which is next unfolded to provide a natural view of the colon; and 
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- the "view point" technique, wherein from each view point one view in one 
direction is calculated as in real endoscopy; 

- the "stretched path" technique, wherein the colon wall is projected onto the 
walls of a cylinder, which is next unfolded and stretched. 

5 All techniques result in a segmentation of the colon based on a voxel model 

comprising the data of a volumetric scan that is projected on a flat surface and represented as 
a surface model. Although the images created by means of virtual endoscopy are in the 
relevant field usually referred to as three-dimensional (3D) images, they are in fact more 
accurately described as the image of a two-dimensional projection of a 3D structure on a 

10 surface. Throughout this text these images will be referred to as 3D images. 

The view point technique is a classical technique that is known in the art and 
among others described by Rogalla P, Terwisscha van Scheltinga J, Hamm B (Eds) in 
"Virtual endoscopy and related 3D techniques", Berlin, Springer Verlag (2001). This book is 
part of the series Medical Radiology Diagnostic Imaging edited by: Baert AL, Sartor K, en 

15 YoukerJE. 

The stretched path technique is in more detail described in the following 
article by D.S. Paik, C.F. Beaulieu, R.B. Jeffrey, Jr., C.A. Karadi, S. Napel, "Visualization 
Modes for CT Colonography using Cylindrical and Planar Map Projections." J Comput 
Assist Tomogr 24(2), pages 179-88, 2000. 

20 The unfolded cube technique is in more detail described in the article 

"Quicktime VR- an image based approach to virtual environment navigation", by S.E. Chen, 
SIGGRAPH 95, held on 6-11 August 1995, Los Angeles, California, USA, Conference 
Proceedings, Annual Conference Series, pages 29-38. A medical application of the unfolded 
cube technique is described in the following publication by F. Vos, I. Serlie, R. van Gelder, 

25 F. Post, R. Truyen, F. Gerritsen, J. Stoker, and A. Vossepoel, "A New Visualization Method 
for Virtual Colonoscopy", in Medical Image Computing and Computer- Assisted Intervention 
- MICCAI 2001 (W. J. Niessen and M. A. Viergever, eds.), pp. 645-654, Springer-Verlag, 
October 2001. 

According to the unfolded cube technique two-dimensional images resulting 
30 from the rendering techniques mentioned above are projected or mapped onto a plane or the 
sides of an object The object is next unfolded in two-dimensions and stored. The resulting 
stored two-dimensional unfolded object image simulates a three-dimensional view from a 
viewpoint from within the object. 
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Various different objects can be used to map the environment on, such as a 
cube, a sphere, a cylinder, etc. Suitable projecting or mapping techniques are known per se in 
the field of computer graphics. The two-dimensional unfolded object images can be stored in 
different formats, for instance a general format, such as jpig or a specific format, such as a 
5 Quicktime VR-format. The resulting two-dimensional unfolded object images comprise a lot 
of information, but are suitable for display on a low-end computer. 

An example of a stored two-dimensional unfolded object image resulting from 
the above projecting is shown in Fig. 2. Image 1 shows the inner colon wall of a patient. As 
can be clearly seen the originally two-dimensional images simulating the three-dimensional 
1 0 view are mapped on the sides of a cube 2, which is then folded open in two dimensions. The 
six sides of the cube are denoted as front (F), left (L), right (R), back (B), up (U) and down 
(D). This way the user is given an intuitive representation of the originally 3D information. 

An image as shown in Fig. 2 can be calculated for a number of view points 
lying on a view path that is defined through the colon by means of known techniques. 
15 Preferably the calculated two-dimensional images are displayed in a sequence. This way the 
reviewer is given the impression to be immersed in the three-dimensional environment as if 
he or she could "fly through it". A virtual environment based on the stored two-dimensional 
images can be calculated at one or more viewpoints in the three-dimensional view allowing 
interactive inspection of the virtual environment. This embodiment provides the user the 
20 possibility to dynamically inspect the environment of a viewpoint interactively. 

However the image 1 according to the state of the art is a monoscopic image. 
According to the method of the invention stereoscopic images are calculated and shown thus 
providing the human observer a natural impression of the patient data including all relevant 
geometrical information, such as depth and height information. 
25 The method according to the invention thus comprises for each image the 

following steps 20 through 50. Three different preferred embodiments of the method 
according to the invention are illustrated in figs. 3 trough 5. All three figures show a picture 
of a colon 3 wherein a view path 4 is defined using known techniques. View points mi (for 
i=l . . .n, wherein n is the number of view points) represent positions on the view path for 
30 which monoscopic images Mi are generated according to the method of the state of the art. 
Viewpoints l\ and r\ are positions for generating left and right eye images Li and Rj to form a 
stereoscopic image according to the method of the invention. 

Step 20: Calculating an image for the left eye from a first viewpoint. 
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As mentioned above several techniques are known in the field to calculate a 
three-dimensional monoscopic image Li from a first viewpoint lj that is suitable for the left 
eye. When using the unfolded cube technique an image like image 1 can for instance be used 
as left eye image. 

5 Step 30:Calculating an image for the right eye from a second viewpoint that 

differs from the first viewpoint. 

Once a method is chosen to calculate a monoscopic left eye image Li, the same 
method will be used to calculate a three-dimensional monoscopic right eye image Rj. The 
right eye image is to be calculated from a second view point r* that differs from the first view 
10 point li. 

In order to create a natural view the view direction from both viewpoints li, ri 
preferably is essentially parallel. However an image pair may also be calculated with 
converging view directions, in which case image distortions are created. 

For the distance between both view points preferably the general rule applies 

1 5 that the distance between the view points should be approximately 1/30 part of the distance 
from the view points to the object(s) to be visualized, for instance the colon wall. In practice 
this leads to a preferred distance between the view points of one or more millimeters that may 
lie in the range of 1 - 5 millimeters, and more preferably in the range of 2 - 3 millimeters. 
General information about stereoscopic imaging can be found in: Jacobus G. Ferwerda, "The 

20 World of 3-D, A Practical Guide to Stereo Photography", 1982. 

Several approaches can be chosen to define the first and second view points lj, 
n, which will be discussed below as part of three preferred embodiments. 

Step 40:Combining the left eye image and the right eye image into a pair to 
form a stereoscopic image. 

25 Fig. 3 schematically illustrates a first preferred embodiment of the method 

according to the invention. According to this first preferred embodiment for each 
stereoscopic image (Li, Ri) the first view point lj lies left of the view path 4 and the second 
view point rj lies right of the view path 4. Preferably the first viewpoint li lies on a first line 
(not shown) that essentially follows the view path at a certain distance. Preferably the second 

30 view point n lies on a second line (not shown) that essentially follows the view path at a 

certain distance. The first and second lines preferably lie on different sides of the view path. 
According to this embodiment two new monoscopic images Li and R\ need to be generated 
for two view points lj and r\ following the view path 4 to create a stereoscopic image (Li, Ri ) 
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instead of one monoscopic image Mi for view points mi on the view path 4 according to the 
known method. 

Fig. 4 schematically illustrates a second preferred embodiment of the method 
according to the invention. For each stereoscopic image at least the first viewpoint li or the 
5 second view point n lies on the view path 4. In fig- 4 the first view point li lies on the view 
path 4 and the second view point n lies right of the view path 4. The first viewpoint li is now 
equal to the viewpoint m\ (see fig. 3) for the method according to the state of the art. 
Preferably the second view point rj lies on a line that essentially follows the view path 4 at a 
certain distance. According to this embodiment two monoscopic images Lj and Rj need to be 

10 generated for two view points li and r i? lying respectively on and next to the view path 4, to 
create a stereoscopic image (Li, R* ). In case the stereoscopic viewing technique is used in 
addition to the known monoscopic viewing technique the already calculated monoscopic 
images Mi for view points mi lying on the view path can be efficiently used as either the left 
or the right eye images in each stereoscopic image. 

15 Fig. 5 schematically illustrates a third preferred embodiment of the method 

according to the invention, wherein for each stereoscopic image (Li, Ri ) both the first view 
point li and the second view point r, lie on the view path 4. The resulting stereoscopic image 
provides a side view perpendicular to the direction of the view path 4. In case the 
stereoscopic viewing technique is used in addition to a known monoscopic viewing technique 

20 wherein monoscopic side view images Mi are already calculated for view points mj lying on 
the view path, these can be efficiently used as both the left and the right eye images in each 
stereoscopic image. In fig, 5 a number of corresponding pairs of viewpoints are indicated (li > 
r\ ) and 0i-i > r i-i ) where x\ « 1m and ru = !«. This applies for instance to the unfolded cube 
technique, discussed above, comprising four side views, respectively, L, R, U and D. The 

25 amount of images to be calculated is now equal to the amount of images to be calculated for 
the known monoscopic viewing technique. 

Step 50: Showing the stereoscopic image using stereoscopic imager means. 
Once the stereoscopic images are generated they now need to be shown to the 
observer. Generally this is performed by the following two steps: Showing the left and right 

30 eye image forming a stereoscopic image with different modification; and Arranging the 

stereoscopic imager means such that the left eye image is passed to the left eye and the right 
eye image is passed to the right eye. 

In the field of computer graphics different techniques to accomplish this type 
of modification are available. Some of them will be discussed below. 
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Firstly the modification can be accomplished by alternately showing the left 
and right eye image of a stereoscopic image with different polarization. A human observer 
may then view the images using stereoscopic imager means that are provided with 
correspondingly differently polarized viewing means for respectively the left and right eye. 
5 In practice the left and right eye images can for instance be alternately shown on a screen at 
high frequency, while the observer for instance wears polarized glasses. 

Secondly the modification can be accomplished by showing the left and right 
eye image of a stereoscopic image with different time-multiplexation. The stereoscopic 
imager means then need to be provided with different viewing means for the left and right 
10 eye, that are to be activated separately by a control unit based on corresponding time- 
multiplexation signals. For this purpose an LCD shutter may be used that is controlled by a 
computer using infrared signals. 

Preferably the stereoscopic images are viewed under the same or like 
conditions under which the stereoscopic images were calculated or recorded. In this respect 
15 the viewing angle and perspective are relevant. By incorporating the viewing means in a 
head-mountable display a geometrically correct presentation of the stereoscopic images can 
be assured. 

Thirdly, as an alternative to the first and second showing techniques the 
stereoscopic imager means may comprise a lenticular screen. Lenticular screens comprise an 
20 array of cylindrically shaped lenses that are able to show a stereoscopic image directly 

without the need for any viewing means to be worn by the observer. Lenticular screens are 
known per se in the art and are for instance described in American Patents US6064424 and 
US61 1 8584, both by the same present applicant. 

It is noted that throughout this text the words left eye and right eye are 
25 interchangeable. 

The method according to the invention is preferably carried out by a system 
for visualizing an internal hollow organ of a subject based on a volumetric scan thereof, 
which systems comprises means for carrying out the steps of the method according to the 
invention. Said means preferably comprise a computer program. Based on the explanation 
30 given herein a skilled person will be able to translate the steps of the method into such a 
computer program to carry out the method. 

The system described can be directly coupled to the data acquisition system 
for acquiring the data of the subject concerned. This data set can be acquired by means of 



PHNL020249EPP 




11 28.03.2002 
various techniques, such as 3D X-ray rotational angiography, computed tomography, 
magnetic resonance imaging or magnetic resonance angiography. 

Summarizing the invention refers to a method of performing virtual endoscopy 
by showing 3D patient data by means of stereoscopic images in order to reveal more detail 
5 thereof. The method is especially developed for medical use to increase the accuracy of 
patient diagnosis. Application of this method results in a virtual image yielding medical 
information, that can be used as an alternative to invasive medical examination methods, 
such as colonoscopy and bronchoscopy. 

The invention is of course not limited to the described or shown embodiment. 
10 The method may be used to visualize surface details of other medical objects, such as blood 
vessels or trachea, and may even be used outside the field of medicine. The invention 
therefore generally extends to any embodiment, which falls within the scope of the appended 
claims as seen in light of the foregoing description and drawings. 
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1 . A medical examination method of visualizing an internal hollow organ of a 
subject based on a volumetric scan thereof, said method comprising the step of reconstructing 
a number of three-dimensional images (1) of the internal surface of the hollow organ (3); 
characterized in that for each image the method comprises the steps of: 

5 - Calculating a three-dimensional image (Li) for the left eye from a first view 

point (li); 

- Calculating a three-dimensional image (Rj ) for the right eye from a second 
view point fa) that differs from the first view point; 

- Combining the left eye image and the right eye image into a pair (Li ,Ri ) to 
1 0 form a stereoscopic image; and 

- Showing the stereoscopic image using stereoscopic imager means. 

2. Method according to claim 1, wherein step a) further comprises the steps of: 

- Defining a view path (4) through the hollow organ (3); and 

15 - Reconstructing the three-dimensional images as seen from view points lying 

on the view path, characterized in that, at least the first (10 or the second view point fa) lies 
on the view path. 

3. Method according to claim 1, wherein step a) further comprises the steps of: 
20 - Defining a view path (4) through the hollow organ (3); and 

- Reconstructing the three-dimensional images as seen from view points lying 
on the view path, characterized in that, both the first (lj) and second view point fa) lie on the 
view path. 



25 4. Method according to claim 3, wherein view points on the view path (4) are 

alternately used as first (10 or second view point fa). 



5. Method according to claim 1, wherein step a) further comprises the step of 

defining a view path (4) through the hollow organ (3), the method being characterized in that, 
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for each image the first view point (10 lies on a first line and the second view point (rO lies on 
a second line, which first and second lines extend essentially parallel to the view path at a 
certain mutual distance. 

5 6. Method according to one or more of the preceding claims, wherein the 

distance between the first (lj) and the second viewpoint (rj) is essentially one or more 
millimeters. 

7. Method according to one or more of the preceding claims, wherein the view 
10 direction in the first (10 and the second view point (r0 is essentially parallel. 

8. Method according to one or more of the preceding claims, wherein step e) 
further comprises the steps of: 

- Showing the left (Li) and right eye image (RO forming a stereoscopic image 
1 5 (Lj, RO with different modification; and 

- Arranging the stereoscopic imager means such that the left eye image is 
passed to the left eye and the right eye image is passed to the right eye. 

9. Method according to claim 8, wherein step I comprises the step of: 

20 - Alternately showing the left (Lj) and right eye image (R0 of a stereoscopic 

image (Li, Ri) with different polarization; and wherein step II comprises the step of: 

- Providing the stereoscopic imager means with correspondingly differently 
polarized viewing means for respectively the left and right eye. 

25 1 0. Method according to claim 8, wherein step I comprises the step of: 

- Showing the left (Lj) and right eye image (Ri) of a stereoscopic image (Lj, R0 
with different time-multiplexation, and wherein step II comprises the step of: 

- Providing the stereoscopic imager means with different viewing means for 
the left and right eye that are to be activated separately by a control unit based on 

3 0 corresponding time-multiplexation signals. 

1 1 . Method according to claim 9 or 10, wherein the viewing means are 

incorporated in a head-mountable display. 
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12. Method according to one or more of the preceding claims 1 through 8, wherein 
the stereoscopic imager means comprise a lenticular screen. 

13. A system for visualizing an internal hollow organ of a subject based on a 
volumetric scan thereof, which systems comprises means for carrying out the steps of the 
method according to one or more of the preceding claims. 



14. Computer program to carry out the method according one or more of the 

preceding claims 1 through 12. 



15 F.PO - 1 28.03.2002 

2 9. G3. 2P07. 

The invention concerns a medical examination method of visualizing an 
internal hollow organ (3) of a subject based on a volumetric scan thereof A number of three- 
dimensional images of the internal surface of the hollow organ are reconstructed For each 
image an image (Li) is calculated for the left eye from a first view point 1*. Next an image for 
5 the right eye (Ri) is calculated from a second view point t\ that differs from the first view 
point. The left eye image and the right eye image are combined into a pair (L b Ri) to form a 
stereoscopic image that is shown using stereoscopic imager means. 

The invention also refers to a system and a computer program to carry out the 
method according to the invention. 
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